Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[e2e] Support managed cluster upgrade testing #619

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

eromanova
Copy link
Member

@eromanova eromanova commented Nov 8, 2024

Closes #313. Partially contains the implementation for #641.

This PR contains the following changes that are not related to the upgrade itself but were required:

  1. Create the default ClusterTemplateChain as part of the hmc-templates helm chart with all the supported cluster templates defined
  2. E2e tests are now applying the default access rules using the default ClusterTemplateChain in the TemplateManagement which allows us to test cluster templates' distribution system from the system namespace to the default namespace. The default access rules are applied on kind and on the standalone clusters
  3. The managed cluster and credentials in e2e testing are deployed in the default namespace instead of the system namespace
  4. Initial support to provide the e2e testing configuration
  5. If the template was not provided in the configuration, automatically find available templates of the requested type from all the available (checking all available template chains) to deploy the managed cluster with it

Changes regarding the e2e upgrade testing:

  1. The managed cluster upgrade will be triggered on the cluster in case the testing configuration contains upgrade: true parameter under the cluster configuration section. It is disabled by default.
  2. If an upgrade was requested, the managed cluster will be upgraded from the source template to the target template provided in the testing configuration. If templates were not provided (either the source, or the target), the latest templates will be automatically found from the available templates and their upgrade sequences. If there are no templates of the particular type or there are no upgrades available, the test will fail.
  3. The upgrade procedure includes changing the spec.template in the ManagedCluster object, verifying that the helm objects were updated, and verifying the cluster health after it

The example of the e2e testing configuration introduced in this PR:

aws:
  standalone: // contains the configuration for the aws standalone cluster testing
    upgrade: true // if set to false - the cluster will be deployed without the upgrade, if set to true - the upgrade will be tested
    template: aws-standalone-cp-0-0-3 // the name of the template to use for the cluster deployment, will be automatically populated if unset
    upgradeTemplate: aws-standalone-cp-0-0-4 // the name of the template to upgrade to, will be automatically populated if unset
  hosted: // contains the configuration for the aws hosted cluster testing
    template: aws-hosted-cp-0-0-3
azure:
  standalone:  // contains the configuration for the azure standalone cluster testing
    upgrade: true
  hosted: // contains the configuration for the azure hosted cluster testing
    upgrade: false
vsphere:
  standalone:  // contains the configuration for the vsphere standalone cluster testing
    upgrade: true
  hosted: // contains the configuration for the vsphere hosted cluster testing
    upgrade: false

To pass the custom testing configuration to the e2e tests you should define the E2E_CONFIG_B64 env var with the e2e configuration in the yaml format encoded in base64 and run make test-e2e.

Out of scope:

  1. This PR does not add any new Templates, or upgrade sequences for the old ones.
  2. Changes in the workflow to support providing the testing configuration (in the scope of Rework providers e2e testing trigger #642)
  3. The testing configuration was implemented in a limited way mainly to support enabling an upgrade for the particular cluster and will be extended later in the scope of Support to provide e2e testing configuration #641

@eromanova eromanova force-pushed the upgrade-testing branch 5 times, most recently from a5d2159 to 2ef6863 Compare November 15, 2024 19:28
@eromanova eromanova marked this pull request as ready for review November 15, 2024 19:29
@eromanova eromanova changed the title [e2e] Managed cluster upgrade testing [e2e] Support managed cluster upgrade testing Nov 15, 2024
@eromanova eromanova marked this pull request as draft November 15, 2024 20:06
@eromanova eromanova marked this pull request as ready for review November 15, 2024 20:32
@eromanova eromanova self-assigned this Nov 21, 2024
@zerospiel
Copy link
Member

I have not reviewed the PR yet, so the general question is: should we enable the e2e job by adding the corresponding label to the PR or are we waiting for the CR/rebases/probably other PRs to be merged beforehand?

@eromanova
Copy link
Member Author

@zerospiel The activities related to reworking the trigger for the e2e testing are out of the scope of this task. Here is a separate task for it: Rework providers e2e testing trigger.

However, I plan to refactor the code related to the testing configuration as part of Support to provide e2e testing configuration, which is currently in progress. Ideally, we should finalize and merge the PR for Support to provide e2e testing configuration first, then rebase and merge this PR afterward.

I would prefer to follow this sequence. Alternatively, this PR can be reviewed and merged now, and the configuration options can be reworked at a later stage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

Add automated tests for managed clusters upgrades
2 participants